Advanced video surveillance systems typically perform many basic analytic functions, such as time recording, motion detection and motion tracking. More advanced features for video surveillance and monitoring are also in high demand. One such function is classifying object that are detected and tracked. Having an accurate object classification function enables users to define more specific events of interest, such as a moving red car, and consequently provides more video storage space, without storing more broadly defined events, such as-motion only. Current object classification methods are not robust and do not offer any flexibility.